Identifying presence of substrates

ABSTRACT

A method for identifying the presence of at least one adulterant substance in a sample. The method comprises receiving sets of sample spectral data, reference spectral data, validation spectral data each set for a respective validation example, and adulterant substance spectral data for said at least one adulterant substance. From these residue data which is representative of a residue which would remain after performing a least squares fitting process between the sample spectral data and the reference spectral data is determined and modified sample residue data which is representative of a residue which would remain after performing a least squares fitting process between the sample spectral data, the reference spectral data and the adulterant substance spectral data is determined. The corresponding two residue data sets are also determined for each validation example. The method then includes performing least one comparison amongst the sample residue data, the modified sample residue data, the validation residue data, and the modified validation residue data; and determining a likelihood value for the presence of said at least one adulterant substance in said sample in dependence on said at least one comparison; and outputting said likelihood value.

This application is a continuation of application Ser. No. 14/913,782filed Feb. 23, 2016, which is a National Phase of InternationalApplication No. PCT/GB2014/000291 filed Jul. 25, 2014, which claimspriority to Great Britain Patent Application No. 1315195.6 filed Aug.23, 2013, the entire contents of each of which are hereby incorporatedby reference.

This invention relates to the identification of the presence ofsubstances. In particular it relates to the identification of at leastone adulterant substance in a sample.

It is well known that in many instances a material can be identifiedfrom its spectrum, be it an optical spectrum, a mass spectrum, or othertype of spectrum. It is often the case that the spectrum of a mixture ofmaterials is a linear combination of the spectra of the individualmaterials, allowing the quantitative composition of the mixture to bedetermined by matching its spectrum to a linear combination of spectraof known materials. Mathematically this match is achieved byminimisation of the numerical differences between the material spectrumand trial combinations chosen from among the known spectra, selectingthe best fit as representing the likely composition of the mixture.

The most commonly used measure of numerical difference in this type ofwork is the Euclidean norm and this process of matching is often knownas a least-squares fit.

It is also well known that the spectra of nominally identical materialscan differ slightly for a variety of reasons. The most fundamentalreason is that samples of what is supposed to be identical material mayactually differ slightly in composition or physical form but to this canbe added systematic measurement differences—deviations from idealperformance in the measuring instrument such as baseline drift andresolution effects—as well as the inevitable random noise thataccompanies all measurements.

The systematic differences between spectra of nominally the samematerial often have distinct spectral character. Consequently it may bethat the different criteria can be substantially refined by consideringthe shape of any difference spectrum. A common way to achieve this isthrough principal component analysis (PCA). This well-known algorithmcan examine the spectral differences (usually differences from the meanspectrum) to discover forms of correlated variation among the data thatoccur in more than one difference spectrum. The result is a series ofspectral forms known as factors of descending significance that, insuitable combinations, describe the various spectral characters of thedifferences.

At the lower significance levels these factors blend into the randomnoise and it is expedient to consider only the most significant factorsand lump the rest together under the heading of noise (the residue).Note that individual factors may not recognizably relate directly tomain sources of differences because the factors are most likelycombinations of such sources.

A particular issue today is being able to determine whether a materialoffered for sale is actually the material intended to be purchased orwhether it is perhaps that material but with the addition of someadulterant. Adulterants in general are added to material in order toreduce the cost and increase the probability in its sale.

For a particular type of substance or sample there will typically be anumber of known potential adulterants which could be used in this waywithout it being immediately obvious to the purchaser.

It may of course be possible to determine the inclusion of suchadulterants in a product for purchase by time consuming and expensivewet chemical analysis techniques but what is desired is a quick andsimple and preferably non-invasive/non-destructive analysis techniquewhich may be used on samples to determine whether they are a “good”unadulterated material or one which has had adulterant substances addedto it.

The present invention is aimed at addressing such a need.

One area where this is of particular interest is that of food stuffs.One can consider cheese as an example. It is possible that cheese may beadulterated by the addition of, say, non-milk protein to bulk out thecheese at a lower expense than the genuine ingredients. At the same timeit will be appreciated that there can be significant variations in thecontent of cheese and thus any analysis technique needs to be able toalert the user to the likely presence of an adulterant substance in asample whilst lowing normal variations which may be present in cheese tostill be considered as cheese by the analysis method.

Another example may be a ground spice which could be potentially bebulked out by another finely ground material such as brick dust.

Whilst in principle the techniques discussed below might be used withspectral data of many different types as alluded to above, in thepresent case optical spectroscopy techniques are of particular interestand, for example, infrared spectroscopy is one particular technique thatmay be used in order to obtain the necessary spectra for use in thepresent techniques.

According to a first aspect of the present invention there is provided amethod for identifying the presence of at least one adulterant substancein a sample comprising the steps of:

receiving a set of sample spectral data acquired for a sample,

receiving a set of reference spectral data,

receiving a plurality of sets of validation spectral data each set for arespective validation example,

receiving a set of adulterant substance spectral data for said at leastone adulterant substance;

determining sample residue data which is representative of a residuewhich would remain after performing a least squares fitting processbetween the sample spectral data and the reference spectral data;

determining modified sample residue data which s representative of aresidue which would remain after performing a least squares fittingprocess between the sample spectral data, the reference spectral dataand the adulterant substance spectral data;

for each validation example, determining validation residue data whichis representative of a residue which would remain after performing aleast squares fitting process between the validation spectral data forthe respective example and the reference spectral data;for each validation example, determining modified validation residuedata which is representative of a residue which would remain afterperforming a least squares fitting process between the validationspectral data for the respective example, the reference spectral dataand the adulterant substance spectral data;performing at least one comparison amongst the sample residue data, themodified sample residue data, the validation residue data, and themodified validation residue data;determining a likelihood value for the presence of said at least oneadulterant substance in said sample in dependence on said at least onecomparison; and outputting said likelihood value.

This allows the identification of the presence of an adulterantsubstance in a sample where there may be some considerable variation inthe character of the sample, and its resulting spectral data, withoutthis being indicative of the presence of an adulterant substance.

The reference spectral data is representative of a substance which isexpected to be nominally the same as the sample if the sample does notcontain adulterant substances.

The validation spectral data is different from the reference spectraldata but still representative of a substance which is expected to benominally the same as the sample if the sample does not containadulterant substances.

One can consider there to be classes of substance. In such a case thereference spectral data and the validation spectral data can be chosento be representative of substances in the same class as the sample.

An example of a class of substance might be cheese. Thus if the sampleis cheese, the reference spectral data and the validation spectral datacan be chosen to be representative of cheese. Depending on thecircumstances the class might be chosen to be smaller. Thus if thesample is a particular type of cheese this might be the class and thereference spectral data and the validation spectral data can be chosento be representative of that type of cheese.

The method can comprise the step of deciding on a class of substance tobe used in dependence on the sample, and selecting the referencespectral data and the validation spectral data so as to berepresentative of substances in that class.

Each of the sample residue data, the modified sample residue data, thevalidation residue data, and the modified validation residue data maycomprise a respective residual spectrum.

Each of the sample residue data, the modified sample residue data, thevalidation residue data, and the modified validation residue data maycomprise a respective scalar value.

The scalar value may comprise the rms of a respective residual spectrum.

The step of performing at least one comparison may comprise,

a) determining a value of a metric in respect of:

i) the sample residue data;

ii) the modified sample residue data;

Ii) the validation residue data for each validation example;

iv) the modified validation residue data for each validation example.

Values for more than one metric may be determined.

The rms of a respective residual spectrum may be chosen as a metric.This may be determined without first determining the spectrum itself asit is an intrinsic part/output of a least squares fitting process.Another possible metric trivially different from rms in this applicationis the standard deviation of the residual spectrum.

Other metrics will generally require determination of the residualspectrum as a first step.

Other possible metrics include:

a ratio of peak-peak amplitude to rms for the residual spectrum,

the number of runs in the residual spectrum, where a run is a contiguousstretch of data that al lies to the same side of zero,

the absolute area under each run,

the root mean square run length.

The step of performing at least one comparison may comprise,

a) determining a value of a metric in respect of:

i) the sample residue data;

ii) the modified sample residue data;

iii) the validation residue data for each validation example;

iv) the modified validation residue data for each validation example,

b) determining a maximum value of the metric for.

i) the validation residue data across the validation examples;

ii) the modified validation residue data across the validation examples,

c) determining an average value of the metric for:

i) the validation residue data across the validation examples;

ii) the modified validation residue data across the validation examples,

d) determining a standard deviation value of the metric for

i) the validation residue data across the validation examples;

ii) the modified validation residue data across the validation examples,

The step of performing at least one comparison may comprise,

a) determining a value of a metric in respect of

i) the sample residue data;

ii) the modified sample residue data;

ii) the validation residue data for each validation example;

iv) the modified validation residue data for each validation example,

b) determining a maximum value of the metric for

i) the validation residue data across the validation examples;

ii) the modified validation residue data across the validation examples,and

c) performing at least one comparison between the value of the metricfor at least one of the sample residue data and the modified sampleresidue on the one hand and the determined maximum value of the metricfor at least one of the validation residue data and the modifiedvalidation residue data on the other hand.

The step of performing at least one comparison may comprise,

a) determining a value of a metric in respect of

i) the sample residue data;

ii) the modified sample residue data;

iii) the validation residue data for each validation example;

iv) the modified validation residue data for each validation example,

b) determining an average value of the metric for

i) the validation residue data across the validation examples;

ii) the modified validation residue data across the validation examples,

c) determining a standard deviation value of the metric for

i) the validation residue data across the validation examples;

ii) the modified validation residue data across the validation examples,and

d) performing at least one comparison between,

the value of the metric for at least one of the sample residue data andthe modified sample residue on the one hand, and

one of:

i) the average for the validation residue data determined in b) plus apredetermined number (n) times the standard deviation for the validationresidue data calculated in c); and

ii) the average for the modified validation residue data determined inb) plus a predetermined number (n) times the standard deviation for themodified validation residue data calculated in c)

on the other hand.

The step of performing at least one comparison may comprise,

a) determining a value of a metric in respect of:

i) the sample residue data;

ii) the modified sample residue data;

ii) the validation residue data for each validation example;

iv) the modified validation residue data for each validation example,

b) calculating the difference between the value of the metric for thesample residue data and the value of the metric for the modified sampleresidue data,

c) for each validation example, calculating the difference between thevalue of the metric for the validation residue data and the value of themetric for the modified validation residue data,

d) determining an average difference between the value of the metric forthe validation residue data and the value of the metric for the modifiedvalidation residue data across the validation examples,

e) determining a standard deviation in the difference between the valueof the metric for the validation residue data and the value of themetric for the modified validation residue data across the validationexamples,

f) comparing the difference between the value of the metric for thesample residue data and the value of metric for the modified sampleresidue data calculated in c) to the average determined in d) plus apredetermined number (n) times the standard deviation calculated in e).

The predetermined number (n) may be chosen to be 3.

This comparison illustrates the degree to which taking a possibleadulterant spectra into account improves the fit to the sample, comparedto the improvement seen for validation examples which should not containthe adulterant. If the improvement for the sample is smaller than theaverage for the validation examples this indicates that the adulterantis not present. If the improvement is significantly greater than theaverage this provides an indication that adulterant is present.

Preferably the step of performing at least one comparison comprisesperforming more than one comparison. Where values of metrics arecalculated these may be used in a plurality of comparisons.

Where more than one comparison is performed the result of eachcomparison may be scored. The step of determining the likelihood valuemay include summing the scores.

The scoring may be as follows:

If the value of the metric for the sample exceeds the average of metricfor the validation examples+n standard deviations, with n chosen to beequal to 3 or greater, score=2.

If the value of the metric for the sample exceeds the determined maximummetric for the validation examples, score=1.

Otherwise score=0.

The method may comprise the further step of estimating the concentrationof a detected adulterant in the sample. This may be done on the basis ofthe size of a magnitude based fitting coefficient used in the fittingprocess.

The method may comprise the further step of determining a significancevalue for a change between the value of a metric in respect of thesample residue data and the modified sample residue data, thesignificance value being calculated as the difference between the valueof a metric in respect of the sample residue data and the modifiedsample residue data divided by 6 times the determined standard deviationof the metric for the validation residue data across the validationexamples.

The method may comprise the further step of outputting an indicator thatthe sample likely includes an adulterant which is distinct from said atleast one adulterant when it is determined that

i) the determined value of a metric in respect of the sample residuedata is greater than the average for the validation residue data plus 3times the standard deviation for the validation residue data; and

ii) the determined value of a metric in respect of the modified sampleresidue data is greater than the average for the modified validationresidue data determined in plus 3 times the standard deviation for themodified validation residue data.

The sample residue data may comprise the rms of a respective residualspectrum which would remain after performing a least squares fittingprocess between the sample spectral data and the reference spectraldata.

The modified sample residue data may comprise the ms of a respectiveresidual spectrum which would remain after performing a least squaresfitting process between the sample spectral data, the reference spectraldata and the adulterant substance spectral data.

The validation residue data may comprise the rms of a respectiveresidual spectrum which would remain after performing a least squaresfitting process between the respective validation spectral data and thereference spectral data.

The modified validation residue data may comprise the rms of arespective residual spectrum which would remain after performing a leastsquares fitting process between the respective validation spectral data,the reference spectral data and the adulterant substance spectral data.

The step of performing at least one comparison amongst the sampleresidue data, the modified sample residue data, the validation residuedata, and the modified validation residue data may comprise comparing atleast one respective pair of the rms values.

The method may comprise:

receiving a set of adulterant substance spectral data for a plurality ofadulterant substances,

determining respective modified sample residue data which isrepresentative of a residue which would remain after performing a leastsquares fitting process between the sample spectral data, the referencespectral data and each set of adulterant substance spectral data, andfor each validation example, determining respective modified validationresidue data which is representative of a residue which would remainafter performing a least squares fitting process between the validationspectral data for the respective example, the reference spectral dataand each set of adulterant substance spectral data.

In such a case the step of performing at least one comparison maycomprise performing at least one comparison in respect of eachadulterant substance; and determining a likelihood value for thepresence of each adulterant substance in said sample in dependence onsaid at least one comparison.

The method may comprise:

receiving a set of adulterant substance spectral data for a plurality ofadulterant substance,

determining respective modified sample residue data which isrepresentative of a residue which would remain after performing a leastsquares fitting process between the sample spectral data, the referencespectral data and at least one selected combination of sets ofadulterant substance spectral data, andfor each validation example, determining respective modified validationresidue data which is representative of a residue which would remainafter performing a least squares fitting process between the validationspectral data for the respective example, the reference spectral dataand the at least one selected combination of sets of adulterantsubstance spectral data.

In such a case the step of performing at least one comparison maycomprise performing at least one comparison in respect of each selectedcombination of adulterant substances; and

determining a likelihood value for the presence of each selectedcombination of adulterant substances in said sample in dependence onsaid at least one comparison.

At least one of the sets of validation spectral data may be based on aspectrum acquired using a spectrometer from a validation examplesubstance.

This leads to validation residue data which is based on examplesubstances.

Example substances with suitable characteristics and variability may behard to come by and thus having an alternative method for determiningvalidation data is useful.

Where the validation residue data comprises a residual spectrum, themethod may comprise the step of creating an additional validationresidue spectrum by the steps of:

computing a discrete wavelet transform of the validation residuespectrum for one validation example;

multiplying at least part of the transform point by point by a normallydistributed sequence of random numbers with unit standard deviation toprovide a modified transform;

performing an inverse of the discrete wavelet transform on the modifiedtransform to produce a spectrum which is usable as an additionalvalidation residue spectrum.

The process may be repeated, with different sequences of random numbersand/or different validation residue data, to produce further additionalvalidation residue spectra.

The or each additional validation residue spectrum may be used todetermine additional validation residue data. This data can be used as,or in the same way as, validation residue data in the processesdescribed above. The additional validation residue data isrepresentative of a residue which would remain after performing a leastsquares fitting process between a respective randomised validationspectrum and the reference spectral data.

In an alternative the process used to create an additional validationresidue spectrum may be used to create additional validation spectraldata by operating on an initial set of validation spectral data ratherthan a residual spectrum. In such a case an additional validationresidue spectrum can then be created by further operating on theadditional set of validation spectral data.

The method may comprise the step of determining modified additionalvalidation residue data which is representative of a residue which wouldremain after performing a least squares fitting process between therespective randomised validation spectrum, the reference spectral dataand the adulterant substance spectral data.

The step of performing at least one comparison, may comprise performingat least one comparison amongst the sample residue data, the modifiedsample residue data, the validation residue data, the modifiedvalidation residue data, the additional validation residue data, and themodified additional validation residue data.

Metrics for the additional validation residue data, and the modifiedadditional validation residue data may be determined as described above,and may be used as described above, in respect of the validation andmodified validation residue data.

Where the expression “the validation residue data for each example” isused above this can comprise the additional validation residue data.

Where the expression “the modified validation residue data for eachexample” is used above this can comprise the additional modifiedvalidation residue data.

It can be useful to compare any improvement seen in taking adulterantsubstances into account against a random result. To facilitate this themethod may comprise generating at least one randomly altered sampleresidue spectrum.

Where the sample residue data comprises a residual spectrum, the methodmay comprise the step of generating at least one randomly altered sampleresidue spectrum by steps of:

computing a discrete wavelet transform of the sample residue data;

multiplying at least part of the transform point by point by a normallydistributed sequence of random numbers with unit standard deviation toprovide a modified transform;

performing an inverse of the discrete wavelet transform operation on themodified transform to produce a spectrum which is usable as a randomlyaltered sample residue spectrum.

The process may be repeated, with different sequences of random numbers,to produce further randomly altered sample residue spectra.

The or each randomly altered sample residue spectrum may be used todetermine randomly altered sample residue data. This can be used as, orin the same way as, validation residue data in the processes describedabove. The randomly altered sample residue data is representative of aresidue which would remain after performing a least squares fittingprocess between a respective randomised spectrum and the referencespectral data.

Further the method may comprise the step of determining modifiedrandomly altered sample residue data which is representative of aresidue which would remain after performing a least squares fittingprocess between the respective randomised spectrum, the referencespectral data and the adulterant substance spectral data.

The step of performing at least one comparison, may comprise performingat least one comparison amongst the sample residue data, the modifiedsample residue data, the validation residue data, the modifiedvalidation residue data, randomly altered sample residue data, andmodified randomly altered sample residue data, and also where present,the additional validation residue data, and the modified additionalvalidation residue data.

Metrics for the randomly altered sample residue data and the modifiedrandomly altered sample residue data may be determined as describedabove, and may be used as described above, in respect of the validationand modified validation residue data.

The randomly altered sample residue data and modified randomly alteredsample residue data can be considered to be a special type of validationresidue data.

The step of performing at least one comparison may comprise,

a) determining a value of a metric in respect of:

i) the sample residue data;

ii) the modified sample residue data;

iii) the randomly altered sample residue data for each randomisedspectrum;

iv) the modified randomly altered sample residue data for eachrandomised spectrum;

and optionally

v) the validation residue data for each validation example;

vi) the modified validation residue data for each validation example.

The step of performing at least one comparison may comprise

a) for each randomised spectrum, calculating the difference between thevalue of the metric for the randomly altered sample residue data and thevalue of the metric for the modified randomly altered sample residuedata,

and at least one of

b) determining an average difference between the value of the metric forthe randomly altered sample residue data and the value of the metric forthe modified randomly altered sample residue data across the data set,

c) determining a standard deviation in the difference between the valueof the metric for the randomly altered sample residue data and the valueof the metric for the modified randomly altered sample residue dataacross the data set,

d) determining a maximum difference between the value of the metric forthe randomly altered sample residue data and the value of the metric forthe modified randomly altered sample residue data across the data set.

The step of performing at least one comparison may comprise,

a) determining a value of a metric in respect of:

i) the sample residue data;

ii) the modified sample residue data;

iii) the randomly altered sample residue data for each randomisedspectrum;

iv) the modified randomly altered sample residue data for eachrandomised spectrum;

b) calculating the difference between the value of the metric for thesample residue data and the value of metric for the modified sampleresidue data,

c) for each randomised spectrum, calculating the difference between thevalue of the metric for the randomly altered sample residue data and thevalue of the metric for the modified randomly altered sample residuedata,

d) determining an average difference between the value of the metric forthe randomly altered sample residue data and the value of the metric forthe modified randomly altered sample residue data across the data set,

e) determining a standard deviation in the difference between the valueof the metric for the randomly altered sample residue data and the valueof the metric for the modified randomly altered sample residue dataacross the data set,

f) comparing the difference between the value of the metric for thesample residue data and the value of metric for the modified sampleresidue data calculated in c) to the average determined in d) plus apredetermined number (n) times the standard deviation calculated in e).

The step of performing at least one comparison may comprise,

a) determining a value of a metric in respect of:

i) the sample residue data;

ii) the modified sample residue data;

iii) the randomly altered sample residue data for each randomisedspectrum;

iv) the modified randomly altered sample residue data for eachrandomised spectrum;

b) calculating the difference between the value of the metric for thesample residue data and the value of metric for the modified sampleresidue data,

c) for each randomised spectrum, calculating the difference between thevalue of the metric for the randomly altered sample residue data and thevalue of the metric for the modified randomly altered sample residuedata,

d) determining a maximum difference between the value of the metric forthe randomly altered sample residue data and the value of the metric forthe modified randomly altered sample residue data across the data set,

e) comparing the difference between the value of the metric for thesample residue data and the value of metric for the modified sampleresidue data calculated in c) to the maximum difference determined ind).

The steps of determining sample residue data, determining modifiedsample residue data, determining validation residue data, determiningmodified validation residue data may be carried out by directlyperforming the respective least squares fitting processes.

However this is computationally very intensive. Thus a differentapproach is preferred.

Thus the method may comprise the steps of.

developing a principal components analysis model of a calibration set ofdata to produce a set of principal factors which represent the set ofreference spectral data;

projecting the principal factors out of the sample spectral data toleave the sample residue data;

projecting the principal factors out of the validation spectral dataeach set for a respective validation example to leave the validationresidue data;

projecting the principal factors out of the adulterant substancespectral data for said at least one adulterant substance to leaveadulterant residue data;

least squares fitting the sample residue data with the adulterantresidue data to generate the modified sample residue data;

least squares fitting the validation residue data with the adulterantresidue data to generate the modified validation residue data.

Where there are a plurality of sets of spectral data for differentadulterant substances, combinations of the adulterant residue data forthe respective substances may be used in the least squares fittingprocesses to generate the appropriate modified sample residue data andmodified validation residue data.

The method may comprise the step of pre-processing the sample spectraldata, the reference spectral data and the validation spectral databefore the steps of determining the residue data.

Where the method includes developing a principal components analysismodel of a calibration set of data, the calibration set of data may bepre-processed before the step of developing a principal componentsanalysis model.

The pre-processing may comprise any one or any combination of:

ensuring that the spectra have the same start and endwavenumber/wavelength and the same data interval,

digital filtering,

weighting,

baseline suppression,

projecting out of unwanted effects.

The step of projecting out unwanted effects may comprise producing anorthonormal set of factor spectra from spectra of the unwanted effectsand subtracting out a scaled amount of each factor from the spectraldata for the sample, the validation examples and the adulterantsubstances.

The unwanted effects may include one or more of baseline effects, anaverage spectrum calculated from the spectra of the validation examples.

The spectral data used in the method may be acquired using one of anumber of different forms of spectral analysis. Infrared spectroscopy,for example, near infrared spectroscopy and particular near infrareddiffuse reflectance spectroscopy is one form of spectral analysis whichis particularly suitable.

According to a second aspect of the present invention there is provideda method for identifying the presence of at least one adulterantsubstance in a sample comprising the steps of:

receiving a set of sample spectral data acquired for a sample,

receiving a plurality of sets of calibration spectral data for use ingenerating a set of reference spectral data, each set of calibrationspectral data being for a respective calibration example,

receiving a plurality of sets of validation spectral data, each set fora respective validation example,

receiving a set of adulterant substance spectral data for said at leastone adulterant substance;

developing a principal components analysis model of the calibration setsof data to produce a set of principal factors which represent the set ofreference spectral data;

projecting the principal factors out of the sample spectral data toleave sample residue data;

projecting the principal factors out of each set of validation spectraldata to leave validation residue data for each validation example;

projecting the principal factors out of the adulterant substancespectral data for said at least one adulterant substance to leaveadulterant residue data;

least squares fitting the sample residue data with the adulterantresidue data to generate modified sample residue data, which representsan effect of taking the adulterant spectral data into account in theprincipal components analysis model; least squares fitting thevalidation residue data with the adulterant residue data to generate themodified validation residue data, which represents an effect of takingthe adulterant spectral data into account in the principal componentsanalysis model,performing at least one comparison amongst the sample residue data, themodified sample residue data, the validation residue data, and themodified validation residue data;determining a likelihood value for the presence of said at least oneadulterant substance in said sample in dependence on said at least onecomparison; and outputting said likelihood value.

According to a third aspect of the present invention there is provided aspectrometer including an analysis module arranged for identifying thepresence of at least one adulterant substance in a sample using a methodas defined above.

According to a fourth aspect of the present invention there is provideda spectrometer arranged for identifying the presence of at least oneadulterant substance in a sample,

the spectrometer having means arranged to:

acquire a set of sample spectral data for a sample,

determine or receive a set of reference spectral data,

acquire or receive a plurality of sets of validation spectral data eachset for a respective validation example,

acquire or receive a set of adulterant substance spectral data for saidat least one adulterant substance;

determine sample residue data which is representative of a residue whichwould remain after performing a least squares fitting process betweenthe sample spectral data and the reference spectral data;

determine modified sample residue data which is representative of aresidue which would remain after performing a least squares fittingprocess between the sample spectral data, the reference spectral dataand the adulterant substance spectral data;

for each validation example, determine validation residue data which isrepresentative of a residue which would remain after performing a leastsquares fitting process between the validation spectral data for therespective example and the reference spectral data;for each validation example, determine modified validation residue datawhich is representative of a residue which would remain after performinga least squares fitting process between the validation spectral data forthe respective example, the reference spectral data and the adulterantsubstance spectral data;perform at least one comparison amongst the sample residue data, themodified sample residue data, the validation residue data, and themodified validation residue data;determine a likelihood value for the presence of said at least oneadulterant substance in said sample in dependence on said at least onecomparison; and output said likelihood value.

According to a fifth aspect of the present invention there is provided acomputer arranged under the control of software for processing spectraldata to identify the presence of at least one adulterant substance in asample,

the computer arranged to:

receive a set of sample spectral data for a sample,

determine or receive a set of reference spectral data,

receive a plurality of sets of validation spectral data each set for arespective validation example,

receive a set of adulterant substance spectral data for said at leastone adulterant substance;

determine sample residue data which is representative of a residue whichwould remain after performing a least squares fitting process betweenthe sample spectral data and the reference spectral data;

determine modified sample residue data which is representative of aresidue which would remain after performing a least squares fittingprocess between the sample spectral data, the reference spectral dataand the adulterant substance spectral data;

for each validation example, determine validation residue data which isrepresentative of a residue which would remain after performing a leastsquares fitting process between the validation spectral data for therespective example and the reference spectral data;for each validation example, determine modified validation residue datawhich is representative of a residue which would remain after performinga least squares fitting process between the validation spectral data forthe respective example, the reference spectral data and the adulterantsubstance spectral data;perform at least one comparison amongst the sample residue data, themodified sample residue data, the validation residue data, and themodified validation residue data;determine a likelihood value for the presence of said at least oneadulterant substance in said sample in dependence on said at least onecomparison; and output said likelihood value.

According to a sixth aspect of the present invention there is provided acomputer arranged under the control of software for processing spectraldata in accordance with any method as defined above to identify thepresence of at least one adulterant substance in a sample.

According to a seventh aspect of the present invention there is provideda computer program comprising code portions which when loaded and run acomputer cause the computer to carry out the steps of:

receiving a set of sample spectral data for a sample,

determining or receiving a set of reference spectral data,

receiving a plurality of sets of validation spectral data each set for arespective validation example,

receiving a set of adulterant substance spectral data for said at leastone adulterant substance;

determining sample residue data which is representative of a residuewhich would remain after performing a least squares fitting processbetween the sample spectral data and the reference spectral data;

determining modified sample residue data which is representative of aresidue which would remain after performing a least squares fittingprocess between the sample spectral data, the reference spectral dataand the adulterant substance spectral data;

for each validation example, determining validation residue data whichis representative of a residue which would remain after performing aleast squares fitting process between the validation spectral data forthe respective example and the reference spectral data;for each validation example, determining modified validation residuedata which is representative of a residue which would remain afterperforming a least squares fitting process between the validationspectral data for the respective example, the reference spectral dataand the adulterant substance spectral data;performing at least one comparison amongst the sample residue data, themodified sample residue data, the validation residue data, and themodified validation residue data;determining a likelihood value for the presence of said at least oneadulterant substance in said sample in dependence on said at least onecomparison; and outputting said likelihood value.

According to an eighth aspect of the present invention there is provideda computer program comprising code portions which when loaded and run ona computer cause the computer to carry out the steps of any of themethods defined above.

There may be a computer program product comprising a machine readabledata carrier carrying the program of the seventh or eighth aspects ofthe invention.

Note that the subfeatures explained above following the first aspect ofthe invention are equally relevant as subfeatures of the remainingaspects of the invention and could be re-written in full with anynecessary changes in wording, this is only not done in the interests ofbrevity.

Each method defined above may comprise a method of using (preferablyoptical) spectroscopy data to identify the presence of at least oneadulterant substance in a sample.

Each method defined above may comprise a method of processing(preferably optical) spectroscopy data to provide an indication of thelikelihood of the presence of at least one adulterant substance in asample.

According to a ninth aspect of the invention there is provided a methodfor generating a randomised spectrum from an initial spectrum for use inspectral analysis, the method comprising steps of

computing a discrete wavelet transform of the initial spectrum;

multiplying at least part of the transform point by point by a normallydistributed sequence of random numbers with unit standard deviation toprovide a modified transform;

performing an inverse of the discrete wavelet transform operation on themodified transform to produce a spectrum which is usable as a randomlyaltered spectrum.

The process may be repeated, with different sequences of random numbers,to produce further randomly altered spectra.

The initial spectrum may be a residue spectrum. The initial spectrum maybe a sample spectrum. The initial spectrum may be a validation spectrum.

Such spectra may be useful where limited initial spectra are availableand/or where it is desired to investigate the effect of randomvariations in spectra which might be expected to occur due to physicaleffects and variations. More realistic results should be achieved thandirectly randomizing the original spectrum.

Embodiments of the present invention will now be described, by way ofexample only, with reference to the accompanying drawings, in which:

FIG. 1 is a schematic view of a spectrometer arranged for identifyingthe presence of at least one adulterant substance in a sample;

FIG. 2 is a flow chart showing, in highly schematic form, the principlesunderlying the present techniques; and

FIG. 3 is a flow chart showing the steps carried out in one particularimplementation of the present techniques.

The present apparatus and techniques are arranged for identifying thepresence of at least one adulterant substance in a sample. The apparatusand techniques make use of analysing the spectral data from the sample,together with spectral data from other substances, which are nominallythe same as the sample, assuming that it is unadulterated, and also thespectra for known adulterants.

The sample spectra clearly have to be acquired for each sample as andwhen it is to be analysed.

On the other hand, the remaining spectral data may be acquired, at leastpartly, in a one time or periodic collection operation, or alternativelymay be acquired each time analysis of samples is to take place.

As mentioned in the introduction, the present techniques may be used foridentifying the presence of at least one adulterant substance withdifferent types of spectroscopic analysis. Thus, for example, theinitial spectral data may be acquired using an optical spectroscopytechnique or a mass spectroscopy technique, or any other suitablespectroscopy technique, where the resulting spectrum will be indicativeof a particular substance.

At the present time optical spectroscopy based techniques are ofparticular interest infrared spectroscopy being one of those and diffusereflectance near infrared spectroscopy being of particular interest dueto its appropriateness for use in identifying adulterant substanceswhich may be present in foodstuffs samples which is a current area ofparticular interest.

Thus the remainder of this description will be written in terms of asystem and technique relying on use of infrared spectroscopy, but itwill be appreciated that the present techniques are not at al limited tothe use of this type of spectroscopy. Furthermore much of thedescription below will in fact be generic to any and all types ofspectroscopy which might be used for acquiring the spectral data.

FIG. 1 schematically shows a spectrometer 1 for obtaining spectra fromsamples 2. The spectrometer 1 comprises a main body portion 3, ananalysis module 4, and an output device 5.

It will be appreciated that in some circumstances an analysis module 4and/or output device 5 such as a screen or printer may be providedseparately from the spectrometer.

The analysis module 4 typically will comprise a computer (including aspectral data processor 6) which can be used in the analysis anddetermining steps which form part of the present techniques which willbe described in more detail below. When suitably programmed, such acomputer can embody the present invention. Similarly when the analysismodule of the spectrometer is suitably programmed this may also embodythe present invention.

The invention may also be embodied in a computer program which may becarried on a physical data carrier such as a CD, DVD, hard drive, flashdrive or similar machine readable data carrier.

FIG. 2 shows in highly schematic form the basic overall process which isperformed when carrying out the present techniques in embodiments of thepresent invention.

In step 201 spectral data is obtained for a sample, a referencespectrum, a set of validation examples and a plurality of adulterantsubstances. In each case this spectral data may at least start with atleast one spectrum being acquired using the spectrometer 1. Of course,in each case, the spectral data obtained for the sample, referencematerial, validation examples and adulterants will be representative ofthe respective substances.

In step 202 a fit is carried out between the sample spectral data andthe reference spectral data, whereas in step 203 a fit is carried outbetween each set of validation spectral data and the reference spectraldata. Thus step 203 is carried out v times where there are v differentvalidation examples.

In step 204 a fit is carried out between the sample on the one hand andthe reference spectral data and adulterant spectral data on the otherhand. Thus the idea here is to see whether if the spectrum forparticular adulterant is taken into account, this leads to a better fit.To put this another way, the question being asked is whether thatparticular adulterant is in fact present in the sample.

In step 205 a fit is carried out between the validation spectral datafor particular validation example on the one hand, and the referencespectral data and the adulterant spectral data on the other hand. Againthis process is carried out v times where there are v validationexamples.

In step 206 comparisons can be made between the results of the fitscarried out in each of steps 202 to 205. In particular, it is possibleto measure whether the improvement in fit to the sample in step 204 madeby including consideration of the adulterant is greater than/significantin the context of the improvement which is seen when the validationspectral data is fitted to the reference spectral data and thecorresponding adulterant spectral data. As will be appreciated, thisacts as a check on whether any improvement seen in fit to the samplewhen taking this particular adulterant into account is real or just atrivial/chance improvement. This comparison process will be defined inmore detail further below.

After this comparison process has been carried in respect of aparticular adulterant, steps 204 and 205 may be repeated for otheradulterant substances and indeed combinations of adulterant substances.

In theory there is no limit to the number of different adulterants andadulterant combinations which may be considered in a process such asthis. However it will be appreciated that there can be practicalconsiderations to take into account in terms of the amount of dataprocessing which is to be carried out. Thus it may, for example, bepractical to consider combinations of up to three adulterants as beingpresent in any one sample and thus carry out steps 204 and 205 inrespect of fitting the sample and validation data to the reference plusup to three different adulterant substances.

The sample spectral data s self-evidently the spectral data whichrelates to the sample. On the other hand the reference spectral data isspectral data representing a good or clean substance of the same type asthe sample. The reference spectral data may in fact be obtained byprocessing spectra from a number of example substances which are of thenominally same type a the samples to be tested.

Similarly the validation spectral data each relate to spectral datataken from a substance which is nominally the same as the sample.

To give a specific example, if the current techniques are to be used inthe situation where the sample is cheese, then the reference spectraldata will be that corresponding to at least one example of unadulteratedcheese and the validation sets of spectral data will each relate toother unadulterated samples of cheese.

On the other hand, of course, the adulterant spectral data will relateto substances which are known for use as adulterants added to cheese.

It will be appreciated that the present techniques may be used inrelation to many different substances or classes of substances. In eachcase it will be important to obtain reference spectral data andvalidation spectral data in relation to “good” or “clean” examples ofsubstances which are nominally the same or in the same class as that ofthe sample which is to be investigated and similarly to obtainadulterant spectral data which it is known or suspected may be used toadulterate the type of substance of which the sample is an example.

It should be noted that the process described in FIG. 2 is a generalisedconceptual process which underlies the present techniques. The actualimplementation of the present techniques might follow the same stepsshown in FIG. 2 but this is not necessarily the case.

In particular, in the currently preferred implementation of the presenttechniques, a slightly different series of steps is undertaken whichhave the same conceptual result as the steps shown in FIG. 2 but theactual processing steps carded out are different.

FIG. 3 is a flow chart showing a series of steps which are taken inparticular implementation of the present techniques.

In step 301 spectra are obtained. As mentioned above these may all beobtained by the spectrometer 1 which will be used for analysing thesample under test or some of them may be obtained separately/earlier. Animportant part of this first step 301 is to gather consistent digitalspectra. Preferably they are measured under very similar conditions. Thespectra gathered comprise:

-   -   a) spectrum of sample under test    -   b) comparison spectra of known unadulterated material—such        spectra should be representative of all expected sample        variation    -   c) spectra of possible adulterants    -   d) spectra of known unwanted effects (e.g. baseline and water        vapour absorption)—typical baselines can be simulated by        polynomial curves.

Note that all spectra need to be compatible. That is they should havethe same start and end wavenumber/wavelength and the same data interval.Spectra may be shortened/interpolated through a common range to achievethis or indeed to restrict the range of spectrum considered by thecurrent processes. As far as practical, spectra should be measured underthe same conditions to avoid introducing unnecessary inconsistencies.Spectra are assumed to be additive for the purposes of mixture analysis;logarithmic, or other conversion may be required to achieve this.

All of the spectra may be pre-processed in the same way according tochoices made by the user.

Thus in step 302 the spectra may optionally be flittered. Note that thesame digital filter should be used on all spectra. Possible filteringincludes smoothing (low pass filter) to reduce high resolution noise,differentiation (high pass filter) to reduce baseline effects, or acombination (band pass filter) might be the best choice. Further, notchfiltering can be used to reduce periodic noise (e.g. fringes). Fouriertransform filters can be used to tailor response. Discrete waveletfilters can be used to reduce known problem areas.

As a further form of pre-processing the spectra may all be weighted instep 303. Again all spectra should be treated the same, such that ifthere is any weighting, all spectra should be multiplied by the sameweighting spectrum. Weighting can be graded according to the expectedreliability of the spectral region. Low signal to noise regions will begiven low weighting. Low weighting would also be given to regions ofirrelevant variability (e.g. water vapour absorption). Signal to noiseratio could be estimated from replicate spectra or generalconsiderations of spectral energy throughput. In the currentimplementation, weight is never allowed to go to zero but can betruncated at a very low value. This allows the unweighted spectrum to bereconstructed by dividing by the weight spectrum if and when desired.

Following any such optional filtering in step 302 and/or weighting instep 303, in step 304 known unwanted effects are projected out of thedata spectra. That is to say out of each of the spectra listed at a) toc) above in the description of step 301. The step of projecting outknown unwanted effects in the present implementation comprises usingsingular value decomposition or similar to produce an orthonormal set offactor spectra from the pre-processed spectra (d)) of unwanted effects.Then a scaled amount of each factor is subtracted away from all of theremaining spectra i.e. the spectra labeled a) to c) above. The scale isgiven by the scalar product of the spectrum being processed and therespective factor.

Note that it may be advantageous to include the mean of the comparisonspectra (b) in the list above) as an unwanted effect and to subtractthis from all of the data spectra a) to c). This can help laternumerical stability as it means that later processing is carried outonly in respect of differences in the spectra rather than the wholevalue.

Note that this process of projecting out is equivalent to least-squaresfitting of the group of unwanted spectra to the remaining spectra andsubtracting away the fit. It would also be possible to defer the processand instead include the unwanted spectra in the calibration set modelwhich will be explained in more detail below.

In step 305 the comparison spectra (b) mentioned in relation to step301) are separated into calibration and validation sets. This separationis somewhat arbitrary as each of the comparison spectra are chosen to berepresentative of unadulterated substances which are nominally the sameas the samples which are to be considered in due course. A number ofsuch examples are needed in the present techniques to act as acalibration set to allow the generation of the reference spectral data.Further a number are required as validation examples to check whetherany improvement in fit when taking adulterants into account, is a realeffect as discussed in general terms above. The selection of thecalibration and validation sets from the comparison spectra might becarried out by the user or selected randomly by the system. Randomselection by the system is preferred since it will avoid any effectsthat might be introduced by user choice.

In step 306 a Principal Components Analysis (PCA) model of thecalibration set is developed. Production of the PCA model may beaccomplished using singular value decomposition (SVD) or non-lineariterative partial least-squares (NIPALS) algorithms. The number offactors to be used in the model can be determine by checking either thesingular values (eigenvalues) or the model residues. There are variouswell established tests such as Malinowski's indicator function or the Ftest that can be used. The number of factors can also be determined fromthe residues in the validation set as the factors are progressivelyprojected out. For low numbers of calibration spectra all of the factorsmay be needed. In this case the method becomes equivalent toleast-square fitting. Least-square fitting can still be used when thisis not the case but it is expected that the process will become noisier.

Once the principal factors of the PCA model are known these may be usedto process the sample spectrum and validation set. Thus in step 307 theprincipal factors are projected out of the sample spectrum and each ofthe validation set spectra. What remains in each case is the residualspectrum from fitting the calibration model in combination with thespectra of unwanted effects=the sample residue data and the validationresidue data.

At this stage the residual spectra from the sample and for each examplein the validation set may be considered.

In step 306 statistical metrics of the residual spectra are developedwhich may be used in a comparison process later. When deciding whatmetrics to use it is important to beer in mind that initially it is ofinterest whether the sample spectrum is abnormal i.e. likely to resultfrom the sample including an adulterant material. To some degree thiscan be achieved by comparing the residual spectrum of the sample withthe residuals of the validation spectra.

For any metric applied to the validation residual data, because thereare multiple validation examples and therefore multiple validationresidual spectra/data, it is possible to calculate an average for themetric and its standard deviation. A good rule of thumb is that if themetric for the sample spectrum lies outside the range of average +/−3standard deviations (of the validation data) the sample spectrum isabnormal. The value of +/−3 standard deviations is a first choice butsomewhat arbitrary. In some circumstances therefore a larger number ofstandard deviations might be chosen as a measure. This could beparticularly relevant where the are a limited number of validationexamples.

The root mean square value (rms) for each residual spectrum is easilycalculated and serves as a summary statistical metric. Note that thismetric can be computed without actually computing the residual spectrumsince it is an intrinsic part of the least-squares process. Anotherpossible metric trivially different from rms in this application is thestandard deviation of the residual spectrum. Other metrics will requirethe residual spectrum to be first calculated.

In the present implementation the use of the rms value for each residualspectrum is the preferred form of metric. However other metrics might beused. For example the ratio of the peak to peak amplitude of a residueto its rms value will indicate possible structure in the residual.Further the number of runs in a residue indicates possible structure.Residual spectra typically have an average value close to zero. A run isa contiguous stretch of spectral data that all lies to the same side ofzero. Data with structure tends to have fewer runs than random data. Thesum of absolute area under each run can also show the presence ofstructure. The root mean square length of runs can also show thepresence of structure. Thus these other metrics might be chosen in otherimplementations.

Up to this stage in the process no attempt has been made to see whetherincluding adulterant spectra in the fit will lead to an improvement andhence an indication that the adulterant substance might be present inthe sample. This is the next stage of the process.

In step 309 the principal factors from the PCA model are projected outof the adulterant spectra. Then in step 310 the adulterant residue datafrom step 309 is fitted to the sample residue data and validationresidue data obtained in step 307. The result of this gives

i) modified sample residue data which is residue data for the sampletaking into account the effect of the adulterant spectra; and

ii) modified validation residue data which is residue data for eachvalidation example taking into account the adulterant spectra.

The resulting modified sample residue data and modified validationresidue data resulting from this process is essentially equivalent tothat which would be derived if the adulterant spectra were included inthe PCA model. However it is computationally more efficient toseparately project the principal factors out of the adulterant spectrain 309 and then perform a least-squares fit between the adulterantresidue data and the sample residue data and validation residue datarespectively.

Note that there may be a relatively large number of adulterantsubstances (and spectra) to be considered and also a relatively largenumber of validation examples to be considered. Further in the presenttechniques combinations of adulterant substances are considered and thuscombinations of the adulterant residue data must also be considered inthe fitting process of step 310. Thus there can be considerableprocessing required in carrying out step 310. Any trials with a negativefit coefficient in step 310 are rejected since adulterants are added tomaterials, not subtracted. Further, any trial with a fit coefficientless than the user selected threshold may also be rejected.

In step 311 statistical metrics of the modified residual spectra may bedeveloped. Here the same metrics will be used as discussed above inrelation to step 308. However what is being done at this stage is todetermine those metrics for the modified sample residue data andmodified validation residue data where the effect of including theadulterant spectra has been taken into account. Thus at this stage wehave sample residue data, validation residue data for each validationexample, modified sample residue data (taking the effect of adulterantspectra into account) and modified validation residue data (taking theeffect of adulterant spectra into account) for each validation example.

In step 312 the metrics of each of these four types of residue data maybe compared in order to help determine whether the relevant adulterantsubstance is present in the sample. In particular we want to determineif the improvement in the fit (reduction in the size of the residue)found by including the adulterants in the fit is significant. How doesit compare to a random result? How does it compare to the results forthe validation examples?

Thus as one example comparison which may be carried out, the differencein value between the metric for the sample residue data and the modifiedsample residue data may be compared to the corresponding differencebetween the modified validation residue data and the validation residuedata. In particular the average difference in the validation residuedata may be calculated and the standard deviation in the difference ofthe validation residue data may be calculated and the difference insample residue data may be compared with this average plus three timesthe standard deviation. If the change in metric for the sample residuedata is greater than the average plus three standard deviations for thevalidation residue data this signals the likely presence of theadulterant(s) concerned.

In step 313 the likelihood of an adulterant under consideration beingpresent in the sample can be ranked. For each difference in metric threescores may be identified:

-   -   2 if the metric exceeds average +3 standard deviations    -   1 if the metric exceeds the maximum validation value observed    -   0 otherwise

Such scores can be combined to give a rank in terms of likelihood. Thesemight be given labels such as detected, likely, possible, unlikely andnot detected.

In step 314 additional statistics may be computed and reported. Thus,for example, the significance of the change in any metric may beindicated. This may be calculated on the basis of the change in themetric between the sample residue data and the modified sample residuedata compared with six times the standard deviation in the validationresidue data across the whole set of validation examples.

Further the presence of unknown adulterants may be indicated. If thevalue of the metric for the sample residue data and for the modifiedsample residue data exceeds the average plus 3 standard deviations ofthe metric for the validation residue data across the whole validationset then it is likely that unknown adulterants are present in the sampleand this may be indicated.

In step 315 results may be reported to the user concerning identifiedadulterants. This may be subject to a user determined minimum inadulterant concentration and/or adulterant likelihood that should beconsidered.

For any and all adulterant substances which exceed any such set limitsthe following may be reported—the adulterants—the determinedlikelihood—the estimated concentration of each adulterant—the estimatedconcentration detection limit—the value of the metric for the modifiedsample residual data.

In step 316 various spectra may be computed and output for inspection.All spectra will be presented filtered but unweighted. The spectraoutput may include the sample spectrum, the residual sample spectrumignoring adulterants i.e. the sample residue data, the (combined)adulterant spectrum if any reported, the residual sample spectrum withadulterants fitted i.e. the modified sample residue data.

In step 317 a report may be given of the adulterant which best fits withthe data i.e. that giving the lowest value of the metric for themodified sample residue data together with the estimated likelihood ofthis adulterant being actually present. This might be output where thereare various adulterants that could fit or in circumstances where thereis no adulterant that passes the user's minimum threshold setting. Instep 317 there may also be a report on the possible presence of unknownadulterants where the sample data does not compare well with thevalidation data but none of the known adulterant spectra lead to apositive determination of the presence of one of those adulterants.

In a development of the process described above in relation to FIG. 3,then as well as comparing fitting of the adulterant spectra to both thesample data and validation data, consideration is also given to theeffect of fitting the adulterant spectra to a randomised samplespectrum.

This is because when testing the reduction in sample residual spectrummetric as the adulterant fitting is introduced, it is helpful to know ifthe effect is significant when compared to a random result. One way oftesting this is to fit the adulterants to a series of quasi-randomsample residual spectra. Unfortunately sample residual spectra rarelyappear to be random even when all reasonable spectral factors areaccounted for. What is needed is a spectrum which has a rather similardistribution of features and resolutions but is otherwise random.

Thus in the present techniques such spectra may be created.

The method used to create such spectra is to take the sample residuespectra calculated in step 307 and appropriately process this. Thissample residue spectra is operated upon to compute the discrete wavelettransform of the sample residue spectrum. At least of part of theresulting transform is multiplied point by point by a normallydistributed sequence of random numbers with unit standard deviation. Thevery low frequency part of the transform may be left alone (e notmultiplied by random numbers) so that very slowly changing effects inthe spectrum remain unaffected. The inverse transform is then performedon the result to give back a somewhat similar looking spectrum butdiffering randomly in its detail.

This may be repeated with different sequences of random numbers.

Once multiple such randomised sample residue spectra have been generatedthey may be used in a similar way as the validation residue data. Thusmetrics may be calculated for the randomised sample residue data and therandomised sample residue data may be subjected to a least-square fitwith the adulterant residue data to give modified randomised sampleresidue data. Once such additional residue data exists it may be used incomparing the effect of including effect of the adulterants. Inparticular a comparison may be made between the improvement in fit inthe actual sample data found by taking the adulterant into accountcompared with the improvement in fit which occurs for the randomiseddata.

Clearly if the improvement in fit in the randomised data is equal orgreater to that which is found for the real data then any effect for thereal data can be ignored.

Sometimes there may be a relatively low number of validation/calibrationexamples available for producing the calibration set of spectra andvalidation set of spectra. In such cases a number of differentpossibilities are available.

One way is to produce synthesised quasi-random residual spectra from oneor more genuine residual spectra using the same method as just describedabove. Thus in such a case a residual spectra for one of the validationexamples would be taken and a quasi-random synthesised residual spectragenerated by performing a discrete wavelet transform on the initialresidual spectrum, multiplying at least part of the transform point bypoint by a normally distributed sequence of random numbers with unitstandard deviation, and then performing the inverse transform.

Another possibility is to accept that there are relatively low numbersof validation data and increase the number of standard deviations whichare taken into account when performing comparisons. Thus rather than theaverage plus three standard deviations mentioned above, one might chooseto use average +n standard deviations where n is larger than 3 and afunction of a number of validation examples which are available. Thus nmight be chosen to be 4 or 5 say.

An alternative is to use cross validation, for example “leave one out”.In such a case, leaving one comparison spectrum out, a model is builtfrom the remaining spectra and the residual spectrum of the one left outis calculated. Repeating this process for each comparison spectrumresults in as many validation residual spectra as there are comparisonspectra, at the expense of creating as many models. The model used forthe sample spectrum residual is the full model built from all of thecomparison spectrum. Note that it is also possible to leave more thanone spectrum out at a time.

Further the step of generating a quasi-random spectrum may be achievedby picking the validation example with the greatest residue afterprojecting out of the principal factors and subjecting this validationexample spectrum to a discrete wavelet transform, multiplying the resultof the transform point by point by a normally distributed sequence ofrandom numbers with unit standard deviation and then performing theinverse transform to get back to a somewhat similar looking spectrum.This resulting spectrum can then be used as an additional validationspectrum and subjected to the same process of projecting out principalfactors and so on.

It will be recognised that within the processes described above avariety of different comparisons may actually be made and a variety ofdifferent metrics may be used in the decision making process ofdetermining whether a particular adulterant is present in the sample.Further as will be clear from above, a particularly important part ofany such decision making process is comparing the effect for the sampledata and the validation data of including a consideration of theadulterant spectra.

In the applicant's currently preferred scoring system, rms of theresidual spectra is chosen as the metric. As mentioned above, this hasthe advantage that the rms value of the residual spectra comes naturallyout of the fitting processes performed and thus this metric can be usedwithout first having to calculate the residual spectrum itself. This canhelp with processing. Standard deviation could be used instead. At alater stage the residual spectrum of interest can be calculated wherethis is useful for other purposes or for example for reporting to theuser. However t can be particularly advantageous not to have tocalculate the residual spectra in all cases for all of the comparisonswhich are being made in the process of determining which adulterant andcombinations of adulterants might be present.

The applicant's currently preferred scoring system is as follows. Thisis a system for scoring the likelihood of a specific adulterant mixturebeing present and involves the use of the following metrics:

-   -   a) the rms of the sample residual spectrum, including adulterant        i.e. modified sample residue data    -   b) the rms of the sample residual spectrum, excluding adulterant        i.e. sample residue data    -   c) the difference of a) and b)    -   d) the estimated concentration of each mixture adulterant in the        sample    -   e) the rms of each validation residual spectrum, including        adulterant i.e. modified validation residue data, for each        validation example    -   f) the rms of each validation residual spectrum, excluding        adulterant i.e. validation residue data, for each validation        example    -   g) the difference of e) and f) for each validation example    -   h) the estimated concentration of each mixture adulterant in        each validation sample    -   i) the rms difference of each quasi-random synthesized sample        residual spectrum, excluding versus including adulterant i.e.        rms (randomised sample residue data)—rms (modified randomised        sample residue data) for each quasi-random spectrum

Measures e) through h) apply to multiple validation spectra andtherefore have an average value, a maximum observed value and a standarddeviation a leading to a limit value of average+3σ (limit>max).Similarly, measure i) applies to multiple quasi-random sample residualspectra and so have an average, max and limit calculated on the samebasis. With these metrics, the currently employed likelihood scoringalgorithm is:

If a)<max e) and

-   -   b) >limit f) and    -   c) >limit g) and    -   c) >limit i) and    -   d) >limit h) for all adulterant mixture components        then likelihood is detected        else if c) >limit g) and    -   c) >limit i) and    -   d) >limit h) for all adulterant mixture components        then likelihood is likely        else if a) >=max e)        then likelihood is possible        else if a)<max e) and    -   b)<max f) and    -   c)<max g) and    -   c)<maxi) and    -   d)<max h) for all adulterant mixture components        then likelihood is not detected        else likelihood is not likely

Such scoring is repeated for each adulterant in the library and eachallowed adulterant combination (currently limited to combinations of upto 3 adulterants).

This scoring methodology could be used with a different metric, perhapsmost clearly using standard deviation in place of rms, but also othermetrics.

Of course a different scoring methodology could be used and for examplethe limit value might be determined using a number of standarddeviations other than 3 if desired.

The invention claimed is:
 1. A spectrometer arranged for identifying thepresence of at least one adulterant substance in a physical sample, thespectrometer comprising: an analysis module configured to acquire,determine or receive (i) a set of sample spectral data for a physicalsample, (ii) a set of reference spectral data, (iii) a plurality of setsof validation spectral data, each set for a respective validationexample, and a (iv) a set of adulterant substance spectral data for saidat least one adulterant substance; and a spectral data processorconfigured to execute on the analysis module to: (a) determine: (i)sample residue data which is representative of a residue which wouldremain after performing a least squares fitting process between thesample spectral data and the reference spectral data, (ii) modifiedsample residue data which is representative of a residue which wouldremain after performing a least squares fitting process between thesample spectral data, the reference spectral data and the adulterantsubstance spectral data, (iii) for each validation example, validationresidue data which is representative of a residue which would remainafter performing a least squares fitting process between the validationspectral data for the respective example and the reference spectraldata, and, (iv) for each validation example, modified validation residuedata which is representative of a residue which would remain afterperforming a least squares fitting process between the validationspectral data for the respective example, the reference spectral dataand the adulterant substance spectral data; b) perform, at least onecomparison amongst the sample residue data, the modified sample residuedata, the validation residue data, and the modified validation residuedata; and c) determine a likelihood value for the presence of said atleast one adulterant substance in said sample in dependence on said atleast one comparison.
 2. The spectrometer according to claim 1, whereinthe performing of the at least one comparison, by the spectral dataprocessor, comprises, a) determining, by the spectral data processor, avalue of a metric in respect of: i) the sample residue data; ii) themodified sample residue data; iii) the validation residue data for eachvalidation example; and iv) the modified validation residue data foreach validation example.
 3. The spectrometer according to claim 2,wherein rms of a respective residual spectrum is chosen as the metric.4. The spectrometer according to claim 1, wherein the performing of theat least one comparison, by the spectral data processor, comprises, a)determining, by the spectral data processor, a value of a metric inrespect of: i) the sample residue data; ii) the modified sample residuedata; iii) the validation residue data for each validation example; andiv) the modified validation residue data for each validation example, b)determining, by the spectral data processor, a maximum value of themetric for: i) the validation residue data across the validationexamples; and ii) the modified validation residue data across thevalidation examples, c) determining, by the spectral data processor, anaverage value of the metric for: i) the validation residue data acrossthe validation examples; and ii) the modified validation residue dataacross the validation examples, and d) determining, by the spectral dataprocessor, a standard deviation value of the metric for: i) thevalidation residue data across the validation examples; and ii) themodified validation residue data across the validation examples.
 5. Thespectrometer according to claim 1, wherein the performing of the atleast one comparison, by the spectral data processor, comprises, a)determining, by the spectral data processor, a value of a metric inrespect of: i) the sample residue data; ii) the modified sample residuedata; iii) the validation residue data for each validation example; andiv) the modified validation residue data for each validation example, b)determining, by the spectral data processor, a maximum value of themetric for: i) the validation residue data across the validationexamples; and ii) the modified validation residue data across thevalidation examples, and c) performing at least one comparison, by thespectral data processor, between the value of the metric for at leastone of the sample residue data and the modified sample residue on theone hand and the determined maximum value of the metric for at least oneof the validation residue data and the modified validation residue dataon the other hand.
 6. The spectrometer according to claim 1, wherein theperforming of the at least one comparison, by the spectral dataprocessor, comprises, a) determining, by the spectral data processor, avalue of a metric in respect of: i) the sample residue data; ii) themodified sample residue data; iii) the validation residue data for eachvalidation example; iv) the modified validation residue data for eachvalidation example, b) determining, by the spectral data processor, anaverage value of the metric for: i) the validation residue data acrossthe validation examples; ii) the modified validation residue data acrossthe validation examples, c) determining, by the spectral data processor,a standard deviation value of the metric for: i) the validation residuedata across the validation examples; ii) the modified validation residuedata across the validation examples, and d) performing at least onecomparison, by the spectral data processor, between, the value of themetric for at least one of the sample residue data and the modifiedsample residue on the one hand, and at least one of: i) the average forthe validation residue data determined in b) plus a predetermined number(n) times the standard deviation for the validation residue datacalculated in c); and ii) the average for the modified validationresidue data determined in b) plus a predetermined number (n) times thestandard deviation for the modified validation residue data calculatedin c) on the other hand.
 7. The spectrometer according to claim 1,wherein the performing of the at least one comparison, by the spectraldata processor, comprises, a) determining, by the spectral dataprocessor, a value of a metric in respect of: i) the sample residuedata; ii) the modified sample residue data; iii) the validation residuedata for each validation example; and iv) the modified validationresidue data for each validation example, b) calculating, by thespectral data processor, the difference between the value of the metricfor the sample residue data and the value of the metric for the modifiedsample residue data, c) for each validation example, calculating, by thespectral data processor, the difference between the value of the metricfor the validation residue data and the value of the metric for themodified validation residue data, d) determining, by the spectral dataprocessor, an average difference between the value of the metric for thevalidation residue data and the value of the metric for the modifiedvalidation residue data across the validation examples, e) determining,by the spectral data processor, a standard deviation in the differencebetween the value of the metric for the validation residue data and thevalue of the metric for the modified validation residue data across thevalidation examples, f) comparing, by the spectral data processor, thedifference between the value of the metric for the sample residue dataand the value of metric for the modified sample residue data calculatedin c) to the average determined in d) plus a predetermined number (n)times the standard deviation calculated in e).
 8. The spectrometeraccording to claim 1, wherein the spectral data processor is furtherconfigured to execute on the analysis module to: generate, by theanalysis module, an indicator that the sample likely includes anadulterant which is distinct from said at least one adulterant inresponse to a determination, by the spectral data processor, that i) thedetermined value of a metric in respect of the sample residue data isgreater than the average for the validation residue data plus 3 timesthe standard deviation for the validation residue data; and ii) thedetermined value of a metric in respect of the modified sample residuedata is greater than the average for the modified validation residuedata determined in plus 3 times the standard deviation for the modifiedvalidation residue data.
 9. The spectrometer according to claim 1,wherein the analysis module is configured to further receive, a set ofadulterant substance spectral data for a plurality of adulterantsubstances and the spectral data processor is further configured toexecute on the analysis module to: determine, respective modified sampleresidue data which is representative of a residue which would remainafter performing a least squares fitting process between the samplespectral data, the reference spectral data and each set of adulterantsubstance spectral data, and for each validation example, determine,respective modified validation residue data which is representative of aresidue which would remain after performing a least squares fittingprocess between the validation spectral data for the respective example,the reference spectral data and each set of adulterant substancespectral data, wherein the performing of the at least one comparison, bythe spectral data processor, comprises performing, by the spectral dataprocessor, at least one comparison in respect of each adulterantsubstance; determining, by the spectral data processor, a likelihoodvalue for the presence of each adulterant substance in said sample independence on said at least one comparison; and generating, by theanalysis module using the likelihood value for the presence of eachadulterant, a label for each respective adulterant substance indicatinga likelihood that the respective adulterant substance is present in thephysical sample.
 10. The spectrometer according to claim 1, wherein theanalysis module is configured to further receive a set of adulterantsubstance spectral data for a plurality of adulterant substance, and thespectral data processor is further configured to execute on the analysismodule to: determine, respective modified sample residue data which isrepresentative of a residue which would remain after performing a leastsquares fitting process between the sample spectral data, the referencespectral data and at least one selected combination of sets ofadulterant substance spectral data, and for each validation example,determine, respective modified validation residue data which isrepresentative of a residue which would remain after performing a leastsquares fitting process between the validation spectral data for therespective example, the reference spectral data and the at least oneselected combination of sets of adulterant substance spectral data,wherein the performing of the at least one comparison, by the spectraldata processor, comprises performing, by the spectral data processor, atleast one comparison in respect of each selected combination ofadulterant substances; determining, by the spectral data processor, alikelihood value for the presence of each selected combination ofadulterant substances in said sample in dependence on said at least onecomparison; and generating, by the analysis module using the likelihoodvalue for the presence of each adulterant, a label for each respectiveadulterant substance indicating a likelihood that the respectiveadulterant substance is present in the physical sample.
 11. Thespectrometer according to claim 1, wherein the validation residue datacomprises a residual spectrum, and the spectral data processor isfurther configured to execute on the analysis module to: create, anadditional validation residue spectrum by at least: computing, by thespectral data processor, a discrete wavelet transform of the validationresidue spectrum for one validation example; multiplying, by thespectral data processor, at least part of the transform point by pointby a normally distributed sequence of random numbers with unit standarddeviation to provide a modified transform; and performing, by thespectral data processor, an inverse of the discrete wavelet transform onthe modified transform to produce a spectrum which is usable as anadditional validation residue spectrum.
 12. The spectrometer accordingto claim 1, wherein the sample residue data comprises a residualspectrum, and the spectral data processor is further configured toexecute on the analysis module to: generate, at least one randomlyaltered sample residue spectrum by at least: computing, by the spectraldata processor, a discrete wavelet transform of the sample residue data;multiplying, by the spectral data processor, at least part of thetransform point by point by a normally distributed sequence of randomnumbers with unit standard deviation to provide a modified transform;and performing, by the spectral data processor, an inverse of thediscrete wavelet transform operation on the modified transform toproduce a spectrum which is usable as a randomly altered sample residuespectrum.
 13. The spectrometer according to claim 12, the spectral dataprocessor is further configured to execute on the analysis module to:determine, modified randomly altered sample residue data which isrepresentative of a residue which would remain after performing a leastsquares fitting process between the respective randomized spectrum, thereference spectral data and the adulterant substance spectral data. 14.The spectrometer according to claim 13, wherein the performing of the atleast one comparison, by the spectral data processor, comprisesperforming, by the spectral data processor, at least one comparisonamongst the sample residue data, the modified sample residue data, thevalidation residue data, the modified validation residue data, randomlyaltered sample residue data, and modified randomly altered sampleresidue data, and also where present, the additional validation residuedata, and the modified additional validation residue data.
 15. Thespectrometer according to claim 14, wherein the performing of the atleast one comparison, by the spectral data processor, comprises: a) foreach randomized spectrum, calculating, by the spectral data processor,the difference between the value of the metric for the randomly alteredsample residue data and the value of the metric for the modifiedrandomly altered sample residue data, and at least one of: b)determining, by the spectral data processor, an average differencebetween the value of the metric for the randomly altered sample residuedata and the value of the metric for the modified randomly alteredsample residue data across the data set, c) determining, by the spectraldata processor, a standard deviation in the difference between the valueof the metric for the randomly altered sample residue data and the valueof the metric for the modified randomly altered sample residue dataacross the data set, and d) determining, by the spectral data processor,a maximum difference between the value of the metric for the randomlyaltered sample residue data and the value of the metric for the modifiedrandomly altered sample residue data across the data set.
 16. Thespectrometer according to claim 15, wherein the performing of the atleast one comparison, by the spectral data processor, comprises, a)determining, by the spectral data processor, a value of a metric inrespect of: i) the sample residue data; ii) the modified sample residuedata; iii) the randomly altered sample residue data for each randomizedspectrum; and iv) the modified randomly altered sample residue data foreach randomized spectrum; b) calculating, by the spectral dataprocessor, the difference between the value of the metric for the sampleresidue data and the value of metric for the modified sample residuedata, c) for each randomized spectrum, calculating, by the spectral dataprocessor, the difference between the value of the metric for therandomly altered sample residue data and the value of the metric for themodified randomly altered sample residue data, d) determining, by thespectral data processor, an average difference between the value of themetric for the randomly altered sample residue data and the value of themetric for the modified randomly altered sample residue data across thedata set, e) determining, by the spectral data processor, a standarddeviation in the difference between the value of the metric for therandomly altered sample residue data and the value of the metric for themodified randomly altered sample residue data across the data set, andf) comparing, by the spectral data processor, the difference between thevalue of the metric for the sample residue data and the value of metricfor the modified sample residue data calculated in c) to the averagedetermined in d) plus a predetermined number (n) times the standarddeviation calculated in e).
 17. The spectrometer according to claim 16wherein the performing of the at least one comparison, by the spectraldata processor, comprises, a) determining, by the spectral dataprocessor, a value of a metric in respect of: i) the sample residuedata; ii) the modified sample residue data; iii) the randomly alteredsample residue data for each randomized spectrum; and iv) the modifiedrandomly altered sample residue data for each randomized spectrum; b)calculating, by the spectral data processor, the difference between thevalue of the metric for the sample residue data and the value of metricfor the modified sample residue data, c) for each randomized spectrum,calculating, by the spectral data processor, the difference between thevalue of the metric for the randomly altered sample residue data and thevalue of the metric for the modified randomly altered sample residuedata, d) determining, by the spectral data processor, a maximumdifference between the value of the metric for the randomly alteredsample residue data and the value of the metric for the modifiedrandomly altered sample residue data across the data set, and e)comparing, by the spectral data processor, the difference between thevalue of the metric for the sample residue data and the value of metricfor the modified sample residue data calculated in c) to the maximumdifference determined in d).
 18. The spectrometer according to claim 1,wherein the determining of sample residue data, the determining ofmodified sample residue data, the determining of validation residuedata, and the determining of modified validation residue data arecarried out by directly performing, by the spectral data processor, therespective least squares fitting processes.
 19. The spectrometeraccording to claim 1, wherein the spectrometer holds or develops aprincipal components analysis model of a calibration set of data toproduce a set of principal factors which represent the set of referencespectral data; and the spectral data processor is further configured toexecute on the analysis module to: project the principal factors out ofthe sample spectral data to leave the sample residue data; project theprincipal factors out of the validation spectral data, each set for arespective validation example to leave the validation residue data;project the principal factors out of the adulterant substance spectraldata for said at least one adulterant substance to leave adulterantresidue data; least squares fit the sample residue data with theadulterant residue data to generate the modified sample residue data;and least squares fit the validation residue data with the adulterantresidue data to generate the modified validation residue data.
 20. Thespectrometer according to claim 19, wherein the spectrometer is arrangedto hold a plurality of sets of spectral data for different adulterantsubstances, and wherein combinations of the adulterant residue data forthe respective substances are used in the least squares fittingprocesses, by the spectral data processor, to generate the appropriatemodified sample residue data and modified validation residue data.
 21. Aspectrometer arranged for identifying the presence of at least oneadulterant substance in a physical sample, the spectrometer comprising:an analysis module configured to receive (i) a set of sample spectraldata acquired for a sample, (ii) a plurality of sets of calibrationspectral data for use in generating a set of reference spectral data,each set of calibration spectral data being for a respective calibrationexample, (iii) a plurality of sets of validation spectral data, each setfor a respective validation example, (iv) a set of adulterant substancespectral data for said at least one adulterant substance, the analysismodule further programmed to develop a principal components analysismodel of the calibration sets of data to produce a set of principalfactors which represent the set of reference spectral data; and aspectral data processor configured to execute on the analysis module to:a) project the principal factors out of the sample spectral data toleave sample residue data; project the principal factors out of each setof validation spectral data to leave validation residue data for eachvalidation example; project the principal factors out of the adulterantsubstance spectral data for said at least one adulterant substance toleave adulterant residue data; least squares fit the sample residue datawith the adulterant residue data to generate modified sample residuedata, which represents an effect of taking the adulterant spectral datainto account in the principal components analysis model; least squaresfit the validation residue data with the adulterant residue data togenerate the modified validation residue data, which represents aneffect of taking the adulterant spectral data into account in theprincipal components analysis model, b) perform at least one comparisonamongst the sample residue data, the modified sample residue data, thevalidation residue data, and the modified validation residue data; c)determine a likelihood value for the presence of said at least oneadulterant substance in said sample in dependence on said at least onecomparison; wherein the analysis module is programmed to generate usingthe likelihood value, a label for the adulterant substance indicating alikelihood that the adulterant substance is present in the physicalsample.
 22. A spectrometer arranged for identifying the presence of atleast one adulterant substance in a physical sample, the spectrometercomprising: an analysis module configured to acquire, determine orreceive (i) acquire a set of sample spectral data for a physical sampleat an analysis module of the spectrometer, (ii) a set of referencespectral data, (iii) a plurality of sets of validation spectral data,each set for a respective validation example, (iv) a set of adulterantsubstance spectral data for said at least one adulterant substance; anda spectral data processor configured to execute on the analysis moduleto: a) determine (i) sample residue data which is representative of aresidue which would remain after performing a least squares fittingprocess between the sample spectral data and the reference spectraldata; (ii) modified sample residue data which is representative of aresidue which would remain after performing a least squares fittingprocess between the sample spectral data, the reference spectral dataand the adulterant substance spectral data; (iii) for each validationexample, validation residue data which is representative of a residuewhich would remain after performing a least squares fitting processbetween the validation spectral data for the respective example and thereference spectral data; and (iv) for each validation example, modifiedvalidation residue data which is representative of a residue which wouldremain after performing a least squares fitting process between thevalidation spectral data for the respective example, the referencespectral data and the adulterant substance spectral data; b) perform atleast one comparison amongst the sample residue data, the modifiedsample residue data, the validation residue data, and the modifiedvalidation residue data; c) determine a likelihood value for thepresence of said at least one adulterant substance in said sample independence on said at least one comparison; and d) output saidlikelihood valve.
 23. A spectrometer according to claim 1 which is adiffuse reflectance infrared spectrometer.
 24. A spectrometer accordingto claim 1 which comprises a main body portion, the analysis module andan output device.
 25. A spectrometer according to claim 1 wherein theanalysis module is programmed to generate, using the likelihood value, alabel for the adulterant substance indicating a likelihood that theadulterant substance is present in the physical sample.
 26. Aspectrometer according to claim 1 wherein the analysis module isprogrammed to output said likelihood valve.